JBoss Community Archive (Read Only)

WildFly 9

Management tasks

Controlling operation via command line parameters

To start up a WildFly 8 managed domain, execute the $JBOSS_HOME/bin/domain.sh script. To start up a standalone server, execute the $JBOSS_HOME/bin/standalone.sh. With no arguments, the default configuration is used. You can override the default configuration by providing arguments on the command line, or in your calling script.

System properties

To set a system property, pass its new value using the standard jvm -Dkey=value options:

$JBOSS_HOME/bin/standalone.sh -Djboss.home.dir=some/location/WildFly/jboss-as \
    -Djboss.server.config.dir=some/location/WildFly/jboss-as/custom-standalone

This command starts up a standalone server instance using a non-standard AS home directory and a custom configuration directory. For specific information about system properties, refer to the definitions below.

Instead of passing the parameters directly, you can put them into a properties file, and pass the properties file to the script, as in the two examples below.

$JBOSS_HOME/bin/domain.sh --properties=/some/location/jboss.properties
$JBOSS_HOME/bin/domain.sh -P=/some/location/jboss.properties

Note however, that properties set this way are not processed as part of JVM launch. They are processed early in the boot process, but this mechanism should not be used for setting properties that control JVM behavior (e.g. java.net.perferIPv4Stack) or the behavior of the JBoss Modules classloading system.

The syntax for passing in parameters and properties files is the same regardless of whether you are running the domain.sh, standalone.sh, or the Microsoft Windows scriptsdomain.bat or standalone.bat.

The properties file is a standard Java property file containing key=value pairs:

jboss.home.dir=/some/location/WildFly/jboss-as
jboss.domain.config.dir=/some/location/WildFly/custom-domain

System properties can also be set via the xml configuration files. Note however that for a standalone server properties set this way will not be set until the xml configuration is parsed and the commands created by the parser have been executed. So this mechanism should not be used for setting properties whose value needs to be set before this point.

Controlling filesystem locations with system properties

The standalone and the managed domain modes each use a default configuration which expects various files and writable directories to exist in standard locations. Each of these standard locations is associated with a system property, which has a default value. To override a system property, pass its new value using the one of the mechanisms above. The locations which can be controlled via system property are:

Standalone

Property name

Usage

Default value

java.ext.dirs

The JDK extension directory paths

null

jboss.home.dir

The root directory of the WildFly 8 installation.

Set by standalone.sh to $JBOSS_HOME

jboss.server.base.dir

The base directory for server content.

jboss.home.dir/standalone

jboss.server.config.dir

The base configuration directory.

jboss.server.base.dir/configuration

jboss.server.data.dir

The directory used for persistent data file storage.

jboss.server.base.dir/data

jboss.server.log.dir

The directory containing the server.log file.

jboss.server.base.dir/log

jboss.server.temp.dir

The directory used for temporary file storage.

jboss.server.base.dir/tmp

jboss.server.deploy.dir

The directory used to store deployed content

jboss.server.data.dir/content

Managed Domain

Property name

Usage

Default value

jboss.home.dir

The root directory of the WildFly installation.

Set by domain.sh to $JBOSS_HOME

jboss.domain.base.dir

The base directory for domain content.

jboss.home.dir/domain

jboss.domain.config.dir

The base configuration directory

jboss.domain.base.dir/configuration

jboss.domain.data.dir

The directory used for persistent data file storage.

jboss.domain.base.dir/data

jboss.domain.log.dir

The directory containing the host-controller.log and process-controller.log files

jboss.domain.base.dir/log

jboss.domain.temp.dir

The directory used for temporary file storage

jboss.domain.base.dir/tmp

jboss.domain.deployment.dir

The directory used to store deployed content

jboss.domain.base.dir/content

jboss.domain.servers.dir

The directory containing the output for the managed server instances

jboss.domain.base.dir/servers

Other command line parameters

The first acceptable format for command line arguments to the WildFly launch scripts is

--name=value

For example:

$JBOSS_HOME/bin/standalone.sh --server-config=standalone-ha.xml

If the parameter name is a single character, it is prefixed by a single '-' instead of two. Some parameters have both a long and short option.

-x=value

For example:

$JBOSS_HOME/bin/standalone.sh -P=/some/location/jboss.properties

For some command line arguments frequently used in previous major releases of WildFly, replacing the "=" in the above examples with a space is supported, for compatibility.

-b 192.168.100.10

If possible, use the -x=value syntax. New parameters will always support this syntax.

The sections below describe the command line parameter names that are available in standalone and domain mode.

Standalone

Name

Default if absent

Value

--admin-only

-

Set the server's running type to ADMIN_ONLY causing it to open administrative interfaces and accept management requests but not start other runtime services or accept end user requests.

--server-config
-c

standalone.xml

A relative path which is interpreted to be relative to jboss.server.config.dir. The name of the configuration file to use.

--read-only-server-config

-

A relative path which is interpreted to be relative to jboss.server.config.dir. This is similar to --server-config but if this alternative is specified the server will not overwrite the file when the management model is changed. However a full versioned history is maintained of the file.

Managed Domain

Name

Default if absent

Value

--admin-only

-

Set the server's running type to ADMIN_ONLY causing it to open administrative interfaces and accept management requests but not start servers or, if this host controller is the master for the domain, accept incoming connections from slave host controllers.

--domain-config
-c

domain.xml

A relative path which is interpreted to be relative to jboss.domain.config.dir. The name of the domain wide configuration file to use.

--read-only-domain-config

-

A relative path which is interpreted to be relative to jboss.domain.config.dir. This is similar to --domain-config but if this alternative is specified the host controller will not overwrite the file when the management model is changed. However a full versioned history is maintained of the file.

--host-config

host.xml

A relative path which is interpreted to be relative to jboss.domain.config.dir. The name of the host-specific configuration file to use.

--read-only-host-config

-

A relative path which is interpreted to be relative to jboss.domain.config.dir. This is similar to --host-config but if this alternative is specified the host controller will not overwrite the file when the management model is changed. However a full versioned history is maintained of the file.

The following parameters take no value and are only usable on slave host controllers (i.e. hosts configured to connect to a remote domain controller.)

Name          

Function

--backup

Causes the slave host controller to create and maintain a local copy of the domain configuration file

--cached-dc

If the slave host controller is unable to contact the master domain controller to get its configuration at boot, boot from a local copy previously created using --backup. The slave host controller will not be able make any modifications to the domain configuration, but it will be able to launch servers.

Common parameters

These parameters apply in both standalone or managed domain mode:

Name

Function

-b=<value>

Sets system property jboss.bind.address to <value>. See Controlling the Bind Address with -b for further details.

-b<name>=<value>

Sets system property jboss.bind.address.<name> to <value> where name can vary. See Controlling the Bind Address with -b for further details.

-u=<value>

Sets system property jboss.default.multicast.address to <value>. See Controlling the Default Multicast Address with -u for further details.

--version
-v
-V

Prints the version of WildFly to standard output and exits the JVM.

--help
-h

Prints a help message explaining the options and exits the JVM.

Controlling the Bind Address with -b

WildFly binds sockets to the IP addresses and interfaces contained in the <interfaces> elements in standalone.xml, domain.xml and host.xml. (See Interfaces and Socket Bindings for further information on these elements.) The standard configurations that ship with WildFly includes two interface configurations:

<interfaces>
    <interface name="management">
        <inet-address value="${jboss.bind.address.management:127.0.0.1}"/>
    </interface>
    <interface name="public">
       <inet-address value="${jboss.bind.address:127.0.0.1}"/>
    </interface>
</interfaces>

Those configurations use the values of system properties jboss.bind.address.management and jboss.bind.address if they are set. If they are not set, 127.0.0.1 is used for each value.

As noted in Common Parameters, the AS supports the -b and -b<name> command line switches. The only function of these switches is to set system properties jboss.bind.address and jboss.bind.address.<name> respectively. However, because of the way the standard WildFly configuration files are set up, using the -b switches can indirectly control how the AS binds sockets.

If your interface configurations match those shown above, using this as your launch command causes all sockets associated with interface named "public"  to be bound to 192.168.100.10.

$JBOSS_HOME/bin/standalone.sh -b=192.168.100.10

In the standard config files, public interfaces are those not associated with server management. Public interfaces handle normal end-user requests.

Interface names

The interface named "public" is not inherently special. It is provided as a convenience. You can name your interfaces to suit your environment.

To bind the public interfaces to all IPv4 addresses (the IPv4 wildcard address), use the following syntax:

$JBOSS_HOME/bin/standalone.sh -b=0.0.0.0

You can also bind the management interfaces, as follows:

$JBOSS_HOME/bin/standalone.sh -bmanagement=192.168.100.10

In the standard config files, management interfaces are those sockets associated with server management, such as the socket used by the CLI, the HTTP socket used by the admin console, and the JMX connector socket.

Be Careful

The -b switch only controls the interface bindings because the standard config files that ship with WildFly sets things up that way. If you change the <interfaces> section in your configuration to ignore the system properties controlled by -b, then setting -b in your launch command will have no effect.

For example, this perfectly valid setting for the "public" interface causes  -b to have no effect on the "public" interface:

<interface name="public">
   <nic name="eth0"/>
</interface>

The key point is the contents of the configuration files determine the configuration. Settings like -b are not overrides of the configuration files. They only provide a shorter syntax for setting a system properties that may or may not be referenced in the configuration files. They are provided as a convenience, and you can choose to modify your configuration to ignore them.

Controlling the Default Multicast Address with -u

WildFly 8 may use multicast communication for some services, particularly those involving high availability clustering. The multicast addresses and ports used are configured using the socket-binding elements in standalone.xml and domain.xml. (See Socket Bindings for further information on these elements.) The standard HA configurations that ship with WildFly include two socket binding configurations that use a default multicast address:

<socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
<socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>

Those configurations use the values of system property jboss.default.multicast.address if it is set. If it is not set, 230.0.0.4 is used for each value. (The configuration may include other socket bindings for multicast-based services that are not meant to use the default multicast address; e.g. a binding the mod-cluster services use to communicate on a separate address/port with Apache httpd servers.)

As noted in Common Parameters, the AS supports the -u command line switch. The only function of this switch is to set system property jboss.default.multicast.address. However, because of the way the standard AS configuration files are set up, using the -u switches can indirectly control how the AS uses multicast.

If your socket binding configurations match those shown above, using this as your launch command causes the service using those sockets configurations to be communicate over multicast address 230.0.1.2.

$JBOSS_HOME/bin/standalone.sh -u=230.0.1.2
Be Careful

As with the -b switch, the -u switch only controls the multicast address used because the standard config files that ship with WildFly 8 sets things up that way. If you change the <socket-binding> sections in your configuration to ignore the system properties controlled by -u, then setting -u in your launch command will have no effect.

Application deployment

Managed Domain

In a managed domain, deployments are associated with a server group (see Core management concepts). Any server within the server group will then be provided with that deployment.

The domain and host controller components manage the distribution of binaries across network boundaries.

Deployment Commands

Distributing deployment binaries involves two steps: uploading the deployment to the repository the domain controller will use to distribute its contents, and then assigning the deployment to one or more server groups.

You can do this in one sweep with the CLI:

[domain@localhost:9999 /] deploy ~/Desktop/test-application.war
Either --all-server-groups or --server-groups must be specified.

[domain@localhost:9999 /] deploy ~/Desktop/test-application.war --all-server-groups
'test-application.war' deployed successfully.

[domain@localhost:9999 /] deploy --help
[...]

After you've uploaded the binary using the deploy command, it will be available to the domain controller, and assigned to a server group:

[domain@localhost:9999 /] :read-children-names(child-type=deployment)
{
   "outcome" => "success",
   "result" => [
       "mysql-connector-java-5.1.15.jar",
       "test-application.war"
   ]
}

[domain@localhost:9999 /] /server-group=main-server-group/deployment=test-application.war:read-resource
{
   "outcome" => "success",
   "result" => {
       "enabled" => true,
       "name" => "test-application.war",
       "runtime-name" => "test-application.war"
   }
}

You can remove binaries from server groups with the undeploy command:

[domain@localhost:9999 /] undeploy test-application.war --all-relevant-server-groups
Successfully undeployed test-application.war.

[domain@localhost:9999 /] /server-group=main-server-group:read-children-names(child-type=deployment)
{
   "outcome" => "success",
   "result" => []
}

Managing deployments through the web interface provides an alternate, sometimes simpler approach.

Content Repository

Deployments are referenced in the domain configuration file, and co-located with the domain controller:

[...]
<deployments>
   <deployment name="test-application.war"
               runtime-name="test-application.war">
       <content sha1="dda9881fa7811b22f1424b4c5acccb13c71202bd"/>
   </deployment>
</deployments>
[...]
<server-groups>
   <server-group name="main-server-group" profile="default">
       [...]
       <deployments>
           <deployment name="test-application.war" runtime-name="test-application.war"/>
       </deployments>
   </server-group>
</server-groups>
[...]

(See domain/configuration/domain.xml)

The actual binaries are stored in the content subdirectory:

ls domain/content/
  |---/47
  |-----95cc29338b5049e238941231b36b3946952991
  |---/dd
  |-----a9881fa7811b22f1424b4c5acccb13c71202bd

Standalone Server

Deployments on a standalone server work in a similar way to those on managed domains. The main difference is that there are no server group associations.

Deployment Commands

The same CLI commands used for managed domains work for standalone servers when deploying and removing an application:

[standalone@localhost:9999 /] deploy ~/Desktop/test-application.war
'test-application.war' deployed successfully.

[standalone@localhost:9999 /] undeploy test-application.war
Successfully undeployed test-application.war.

File System Deployments

Deployment content (for example, war, ear, jar, and sar files) can be placed in the standalone/deployments directory of the WildFly 8 distribution, in order to be automatically deployed into the server runtime.

Users are encouraged to use the WildFly 8 management APIs to upload and deploy deployment content instead of relying on the deployment scanner that periodically scans the directory, particularly if running production systems.

Deployment Modes

The WildFly 8 filesystem deployment scanner operates in one of two different modes, depending on whether it will directly monitor the deployment content in order to decide to deploy or redeploy it.

Auto-deploy mode:

The scanner will directly monitor the deployment content, automatically deploying new content and redeploying content whose timestamp has changed. This is similiar to the behavior of previous AS releases, although there are differences:

  • A change in any file in an exploded deployment triggers redeploy. Because EE 6 applications do not require deployment descriptors,
    there is no attempt to monitor deployment descriptors and only redeploy when a deployment descriptor changes.

  • The scanner will place marker files in this directory as an indication of the status of its attempts to deploy or undeploy content. These are detailed below.

Manual deploy mode:

The scanner will not attempt to directly monitor the deployment content and decide if or when the end user wishes the content to be deployed. Instead, the scanner relies on a system of marker files, with the user's addition or removal of a marker file serving as a sort of command telling the scanner to deploy, undeploy or redeploy content.

Auto-deploy mode and manual deploy mode can be independently configured for zipped deployment content and exploded deployment content. This is done via the "auto-deploy" attribute on the deployment-scanner element in the standalone.xml configuration file:

<deployment-scanner scan-interval="5000" relative-to="jboss.server.base.dir"
   path="deployments" auto-deploy-zipped="true" auto-deploy-exploded="false"/>

By default, auto-deploy of zipped content is enabled, and auto-deploy of exploded content is disabled. Manual deploy mode is strongly recommended for exploded content, as exploded content is inherently vulnerable to the scanner trying to auto-deploy partially copied content.

Marker Files

The marker files always have the same name as the deployment content to which they relate, but with an additional file suffix appended. For example, the marker file to indicate the example.war file should be deployed is named example.war.dodeploy. Different marker file suffixes have different meanings.

The relevant marker file types are:

File

Purpose

.dodeploy

Placed by the user to indicate that the given content should
be deployed into the runtime (or redeployed if already
deployed in the runtime.)

.skipdeploy

Disables auto-deploy of the content for as long as the file
is present. Most useful for allowing updates to exploded
content without having the scanner initiate redeploy in the
middle of the update. Can be used with zipped content as
well, although the scanner will detect in-progress changes
to zipped content and wait until changes are complete.

.isdeploying

Placed by the deployment scanner service to indicate that it
has noticed a .dodeploy file or new or updated auto-deploy
mode content and is in the process of deploying the content.
This marker file will be deleted when the deployment process
completes.

.deployed

Placed by the deployment scanner service to indicate that the
given content has been deployed into the runtime. If an end
user deletes this file, the content will be undeployed.

.failed

Placed by the deployment scanner service to indicate that the
given content failed to deploy into the runtime. The content
of the file will include some information about the cause of
the failure. Note that with auto-deploy mode, removing this
file will make the deployment eligible for deployment again.

.isundeploying

Placed by the deployment scanner service to indicate that it
has noticed a .deployed file has been deleted and the
content is being undeployed. This marker file will be deleted
when the undeployment process completes.

.undeployed

Placed by the deployment scanner service to indicate that the
given content has been undeployed from the runtime. If an end
user deletes this file, it has no impact.

.pending

Placed by the deployment scanner service to indicate that it
has noticed the need to deploy content but has not yet
instructed the server to deploy it. This file is created if
the scanner detects that some auto-deploy content is still in
the process of being copied or if there is some problem that
prevents auto-deployment. The scanner will not instruct the
server to deploy or undeploy any content (not just the
directly affected content) as long as this condition holds.

Basic workflows:
All examples assume variable $JBOSS_HOME points to the root of the WildFly 8 distribution.

A) Add new zipped content and deploy it:

  1. cp target/example.war/ $JBOSS_HOME/standalone/deployments

  2. (Manual mode only) touch $JBOSS_HOME/standalone/deployments/example.war.dodeploy

B) Add new unzipped content and deploy it:

  1. cp -r target/example.war/ $JBOSS_HOME/standalone/deployments

  2. (Manual mode only) touch $JBOSS_HOME/standalone/deployments/example.war.dodeploy

C) Undeploy currently deployed content:

  1. rm $JBOSS_HOME/standalone/deployments/example.war.deployed

D) Auto-deploy mode only: Undeploy currently deployed content:

  1. rm $JBOSS_HOME/standalone/deployments/example.war

E) Replace currently deployed zipped content with a new version and deploy it:

  1. cp target/example.war/ $JBOSS_HOME/standalone/deployments

  2. (Manual mode only) touch $JBOSS_HOME/standalone/deployments/example.war.dodeploy

F) Manual mode only: Replace currently deployed unzipped content with a new version and deploy it:

  1. rm $AS/standalone/deployments/example.war.deployed

  2. wait for $JBOSS_HOME/standalone/deployments/example.war.undeployed file to appear

  3. cp -r target/example.war/ $JBOSS_HOME/standalone/deployments

  4. touch $JBOSS_HOME/standalone/deployments/example.war.dodeploy

G) Auto-deploy mode only: Replace currently deployed unzipped content with a new version and deploy it:

  1. touch $JBOSS_HOME/standalone/deployments/example.war.skipdeploy

  2. cp -r target/example.war/ $JBOSS_HOME/standalone/deployments

  3. rm $JBOSS_HOME/standalone/deployments/example.war.skipdeploy

H) Manual mode only: Live replace portions of currently deployed unzipped content without redeploying:

  1. cp -r target/example.war/foo.html $JBOSS_HOME/standalone/deployments/example.war

I) Auto-deploy mode only: Live replace portions of currently deployed unzipped content without redeploying:

  1. touch $JBOSS_HOME/standalone/deployments/example.war.skipdeploy

  2. cp -r target/example.war/foo.html $JBOSS_HOME/standalone/deployments/example.war

J) Manual or auto-deploy mode: Redeploy currently deployed content (i.e. bounce it with no content change):

  1. touch $JBOSS_HOME/standalone/deployments/example.war.dodeploy

K) Auto-deploy mode only: Redeploy currently deployed content (i.e. bounce it with no content change):

  1. touch $JBOSS_HOME/standalone/deployments/example.war

The above examples use Unix shell commands. Windows equivalents are:

cp src dest --> xcopy /y src dest
cp -r src dest --> xcopy /e /s /y src dest
rm afile --> del afile
touch afile --> echo>> afile

Note that the behavior of 'touch' and 'echo' are different but the differences are not relevant to the usages in the examples above.

Deployment overlays

Deployment overlays are our way of 'overlaying' content into an existing deployment, without physically modifying the contents of the deployment archive. Possible use cases include swapping out deployment descriptors, modifying static web resources to change the branding of an application, or even replacing jar libraries with different versions.

Deployment overlays have a different lifecycle to a deployment. In order to use a deployment overlay, you first create the overlay, using the CLI or the management API. You then add files to the overlay, specifying the deployment paths you want them to overlay. Once you have created the overlay you then have to link it to a deployment name (which is done slightly differently depending on if you are in standalone or domain mode). Once you have created the link any deployment that matches the specified deployment name will have the overlay applied.

When you modify or create an overlay it will not affect existing deployments, they must be redeployed in order to take effect

Creating a deployment overlay

To create a deployment overlay the CLI provides a high level command to do all the steps specified above in one go. An example command is given below for both standalone and domain mode:

deployment-overlay add --name=myOverlay --content=/WEB-INF/web.xml=/myFiles/myWeb.xml,/WEB-INF/ejb-jar.xml=/myFiles/myEjbJar.xml --deployments=test.war,*-admin.war --redeploy-affected
deployment-overlay add --name=myOverlay --content=/WEB-INF/web.xml=/myFiles/myWeb.xml,/WEB-INF/ejb-jar.xml=/myFiles/myEjbJar.xml --deployments=test.war,*-admin.war --server-groups=main-server-group --redeploy-affected

Starting & stopping Servers in a Managed Domain

Starting a standalone server is done through the bin/standalone.sh script. However in a managed domain server instances are managed by the domain controller and need to be started through the management layer:

First of all, get to know which servers are configured on a particular host:

[domain@localhost:9999 /] :read-children-names(child-type=host)
{
   "outcome" => "success",
   "result" => ["local"]
}


[domain@localhost:9999 /] /host=local:read-children-names(child-type=server-config)
{
   "outcome" => "success",
   "result" => [
       "my-server",
       "server-one",
       "server-three"
   ]
}

Now that we know, that there are two servers configured on host "local", we can go ahead and check their status:

[domain@localhost:9999 /] /host=local/server-config=server-one:read-resource(include-runtime=true)
{
   "outcome" => "success",
   "result" => {
       "auto-start" => true,
       "group" => "main-server-group",
       "interface" => undefined,
       "name" => "server-one",
       "path" => undefined,
       "socket-binding-group" => undefined,
       "socket-binding-port-offset" => undefined,
       "status" => "STARTED",
       "system-property" => undefined,
       "jvm" => {"default" => undefined}
   }
}

You can change the server state through the "start" and "stop" operations

[domain@localhost:9999 /] /host=local/server-config=server-one:stop
{
   "outcome" => "success",
   "result" => "STOPPING"
}

Navigating through the domain topology is much more simple when you use the web interface.

Controlling JVM settings

Configuration of the JVM settings is different for a managed domain and a standalone server. In a managed domain, the domain controller components are responsible for starting and stoping server processes and hence determine the JVM settings. For a standalone server, it's the responsibility of the process that started the server (e.g. passing them as command line arguments).

Managed Domain

In a managed domain the JVM settings can be declared at different scopes: For a specific server group, for a host or for a particular server. If not declared, the settings are inherited from the parent scope. This allows you to customize or extend the JVM settings within every layer.

Let's take a look at the JVM declaration for a server group:

<server-groups>
       <server-group name="main-server-group" profile="default">
           <jvm name="default">
               <heap size="64m" max-size="512m"/>
           </jvm>
           <socket-binding-group ref="standard-sockets"/>
       </server-group>
       <server-group name="other-server-group" profile="default">
           <jvm name="default">
               <heap size="64m" max-size="512m"/>
           </jvm>
           <socket-binding-group ref="standard-sockets"/>
       </server-group>
</server-groups>

(See domain/configuration/domain.xml)

In this example the server group "main-server-group" declares a heap size of 64m and a maximum heap size of 512m. Any server that belongs to this group will inherit these settings. You can change these settings for the group as a whole, or a specific server or host:

<servers>
       <server name="server-one" group="main-server-group" auto-start="true">
           <jvm name="default"/>
       </server>
       <server name="server-two" group="main-server-group" auto-start="true">
           <jvm name="default">
               <heap size="64m" max-size="256m"/>
           </jvm>
           <socket-binding-group ref="standard-sockets" port-offset="150"/>
       </server>
       <server name="server-three" group="other-server-group" auto-start="false">
           <socket-binding-group ref="standard-sockets" port-offset="250"/>
       </server>
</servers>

(See domain/configuration/host.xml)

In this case, server-two, belongs to the main-server-group and inherits the JVM settings named default, but declares a lower maximum heap size.

[domain@localhost:9999 /] /host=local/server-config=server-two/jvm=default:read-resource
{
   "outcome" => "success",
   "result" => {
       "heap-size" => "64m",
       "max-heap-size" => "256m",
   }
}

Standalone Server

For a standalone sever you have to pass in the JVM settings either as command line arguments when executing the $JBOSS_HOME/bin/standalone.sh script, or by declaring them in $JBOSS_HOME/bin/standalone.conf. (For Windows users, the script to execute is %JBOSS_HOME%/bin/standalone.bat while the JVM settings can be declared in %JBOSS_HOME%/bin/standalone.conf.bat.)

Administrative audit logging

WildFly comes with audit logging built in for management operations affecting the management model. By default it is turned off. The information is output as JSON records.

The default configuration of audit logging in standalone.xml looks as follows:

    <management>
        <security-realms>
...
        </security-realms>
        <audit-log>
            <formatters>
                <json-formatter name="json-formatter"/>
            </formatters>
            <handlers>
                <file-handler name="file" formatter="json-formatter" path="audit-log.log" relative-to="jboss.server.data.dir"/>
            </handlers>
            <logger log-boot="true" log-read-only="true" enabled="false">
                <handlers>
                    <handler name="file"/>
                </handlers>
            </logger>
        </audit-log>
...

Looking at this via the CLI it looks like

[standalone@localhost:9990 /] /core-service=management/access=audit:read-resource(recursive=true)
{
    "outcome" => "success",
    "result" => {
        "file-handler" => {"file" => {
            "formatter" => "json-formatter",
            "max-failure-count" => 10,
            "path" => "audit-log.log",
            "relative-to" => "jboss.server.data.dir"
        }},
        "json-formatter" => {"json-formatter" => {
            "compact" => false,
            "date-format" => "yyyy-MM-dd HH:mm:ss",
            "date-separator" => " - ",
            "escape-control-characters" => false,
            "escape-new-line" => false,
            "include-date" => true
        }},
        "logger" => {"audit-log" => {
            "enabled" => false,
            "log-boot" => true,
            "log-read-only" => false,
            "handler" => {"file" => {}}
        }},
        "syslog-handler" => undefined
    }
}

The audit logging subsystem has a lot of internal dependencies, and it logs operations changing, enabling and disabling its components. When configuring or changing things at runtime it is a good idea to make these changes as part of a CLI batch. For example if you are adding a syslog handler you need to add the handler and its information as one step. Similarly if you are using a file handler, and want to change its path and relative-to attributes, that needs to happen as one step.

JSON Formatter

The first thing that needs configuring is the formatter, we currently support outputting log records as JSON. You can define several formatters, for use with different handlers. A log record has the following format, and it is the formatter's job to format the data presented:

2013-08-12 11:01:12 - {
    "type" : "core",
    "r/o" : false,
    "booting" : false,
    "version" : "8.0.0.Alpha4",
    "user" : "$local",
    "domainUUID" : null,
    "access" : "NATIVE",
    "remote-address" : "127.0.0.1/127.0.0.1",
    "success" : true,
    "ops" : [JMX|WFLY8:JMX subsystem configuration],
        "operation" : "write-attribute",
        "name" : "enabled",
        "value" : true,
        "operation-headers" : {"caller-type" : "user"}
    }]
}

It includes an optional timestamp and then the following information in the json record

Field name

Description

type

This can have the values core, meaning it is a management operation, or jmx meaning it comes from the jmx subsystem (see the jmx subsystem for configuration of the jmx subsystem's audit logging)

r/o

true if the operation does not change the management model, false otherwise

booting

true if the operation was executed during the bootup process, false if it was executed once the server is up and running

version

The version number of the WildFly instance

user

The username of the authenticated user. In this case the operation has been logged via the CLI on the same machine as the running server, so the special $local user is used

domainUUID

An ID to link together all operations as they are propagated from the Doman Controller to it servers, slave Host Controllers, and slave Host Controller servers

access

This can have one of the following values:
*NATIVE - The operation came in through the native management interface, for example the CLI
*HTTP - The operation came in through the domain HTTP interface, for example the admin console
*JMX - The operation came in through the JMX subsystem. See JMX for how to configure audit logging for JMX.

remote-address

The address of the client executing this operation

success

true if the operation succeeded, false if it was rolled back

ops

The operations being executed. This is a list of the operations serialized to JSON. At boot this will be all the operations resulting from parsing the xml. Once booted the list will typically just contain a single entry

The json formatter resource has the following attributes:

Attribute

Description

include-date

Boolan toggling whether or not to include the timestamp in the formatted log records

date-separator

A string containing characters to separate the date and the rest of the formatted log message. Will be ignored if include-date=false

date-format

The date format to use for the timestamp as understood by java.text.SimpleDateFormat. Will be ignored if include-date=false

compact

If true will format the JSON on one line. There may still be values containing new lines, so if having the whole record on one line is important, set escape-new-line or escape-control-characters to true

escape-control-characters

If true it will escape all control characters (ascii entries with a decimal value < 32) with the ascii code in octal, e.g. a new line becomes '#012'. If this is true, it will override escape-new-line=false

escape-new-line

If true it will escape all new lines with the ascii code in octal, e.g. "#012".

Handlers

A handler is responsible for taking the formatted data and logging it to a location. There are currently two types of handlers, File and Syslog. You can configure several of each type of handler and use them to log information.

File handler

The file handlers log the audit log records to a file on the server. The attributes for the file handler are

Attribute

Description

Read Only

formatter

The name of a JSON formatter to use to format the log records

false

path

The path of the audit log file

false

relative-to

The name of another previously named path, or of one of the standard paths provided by the system. If relative-to is provided, the value of the path attribute is treated as relative to the path specified by this attribute

false

failure-count

The number of logging failures since the handler was initialized

true

max-failure-count

The maximum number of logging failures before disabling this handler

false

disabled-due-to-failure

true if this handler was disabled due to logging failures

true

In our standard configuration path=audit-log.log and relative-to=jboss.server.data.dir, typically this will be $JBOSS_HOME/standalone/data/audit-log.log

Syslog handler

The default configuration does not have syslog audit logging set up. Syslog is a better choice for audit logging since you can log to a remote syslog server, and secure the authentication to happen over TLS with client certificate authentication. Syslog servers vary a lot in their capabilities so not all settings in this section apply to all syslog servers. We have tested with rsyslog.

The address for the syslog handler is /core-service=management/access=audit/syslog-handler=* and just like file handlers you can add as many syslog entries as you like. The syslog handler resources reference the main RFC's for syslog a fair bit, for reference they can be found at:
*http://www.ietf.org/rfc/rfc3164.txt
*http://www.ietf.org/rfc/rfc5424.txt
*http://www.ietf.org/rfc/rfc6587.txt

The syslog handler resource has the following attributes:

formatter

The name of a JSON formatter to use to format the log records

false

failure-count

The number of logging failures since the handler was initialized

true

max-failure-count

The maximum number of logging failures before disabling this handler

false

disabled-due-to-failure

true if this handler was disabled due to logging failures

true

syslog-format

Whether to set the syslog format to the one specified in RFC-5424 or RFC-3164

false

max-length

The maximum length in bytes a log message, including the header, is allowed to be. If undefined, it will default to 1024 bytes if the syslog-format is RFC3164, or 2048 bytes if the syslog-format is RFC5424.

false

truncate

Whether or not a message, including the header, should truncate the message if the length in bytes is greater than the maximum length. If set to false messages will be split and sent with the same header values

false

When adding a syslog handler you also need to add the protocol it will use to communicate with the syslog server. The valid choices for protocol are UDP,TCP and TLS. The protocol must be added at the same time as you add the syslog handler, or it will fail. Also, you can only add one protocol for the handler.

UDP

Configures the handler to use UDP to communicate with the syslog server. The address of the UDP resource is /core-service=management/access=audit/syslog-handler=*/protocol=udp. The attributes of the UDP resource are:

Attribute

Description

host

The host of the syslog server for the udp requests

port

The port of the syslog server listening for the udp requests

TCP

Configures the handler to use TCP to communicate with the syslog server. The address of the TCP resource is /core-service=management/access=audit/syslog-handler=*/protocol=tcp. The attributes of the TCP resource are:

Attribute

Description

host

The host of the syslog server for the tcp requests

port

The port of the syslog server listening for the tcp requests

message-transfer

The message transfer setting as described in section 3.4 of RFC-6587. This can either be OCTET_COUNTING as described in section 3.4.1 of RFC-6587, or NON_TRANSPARENT_FRAMING as described in section 3.4.1 of RFC-6587

TLS

Configures the handler to use TLC to communicate securely with the syslog server. The address of the TLS resource is /core-service=management/access=audit/syslog-handler=*/protocol=tls. The attributes of the TLS resource are the same as for TCP:

Attribute

Description

host

The host of the syslog server for the tls requests

port

The port of the syslog server listening for the tls requests

message-transfer

The message transfer setting as described in section 3.4 of RFC-6587. This can either be OCTET_COUNTING as described in section 3.4.1 of RFC-6587, or NON_TRANSPARENT_FRAMING as described in section 3.4.1 of RFC-6587

If the syslog server's TLS certificate is not signed by a certificate signing authority, you will need to set up a truststore to trust the certificate. The resource for the trust store is a child of the TLS resource, and the full address is /core-service=management/access=audit/syslog-handler=*/protocol=tls/authentication=truststore. The attributes of the truststore resource are:

Attribute

Description

keystore-password

The password for the truststore

keystore-path

The path of the truststore

keystore-relative-to

The name of another previously named path, or of one of the standard paths provided by the system. If keystore-relative-to is provided, the value of the keystore-path attribute is treated as relative to the path specified by this attribute

TLS with Client certificate authentication.

If you have set up the syslog server to require client certificate authentication, when creating your handler you will also need to set up a client certificate store containing the certificate to be presented to the syslog server. The address of the client certificate store resource is /core-service=management/access=audit/syslog-handler=*/protocol=tls/authentication=client-certificate-store and its attributes are:

Attribute

Description

keystore-password

The password for the keystore

key-password

The password for the keystore key

keystore-path

The path of the keystore

keystore-relative-to

The name of another previously named path, or of one of the standard paths provided by the system. If keystore-relative-to is provided, the value of the keystore-path attribute is treated as relative to the path specified by this attribute

Logger configuration

The final part that needs configuring is the logger for the management operations. This references one or more handlers and is configured at /core-service=management/access=audit/logger=audit-log. The attributes for this resource are:

Attribute

Description

enabled

true to enable logging of the management operations

log-boot

true to log the management operations when booting the server, false otherwise

log-read-only

If true all operations will be audit logged, if false only operations that change the model will be logged

Then which handlers are used to log the management operations are configured as handler=* children of the logger.

Domain Mode (host specific configuration)

In domain mode audit logging is configured for each host in its host.xml file. This means that when connecting to the DC, the configuration of the audit logging is under the host's entry, e.g. here is the default configuration:

[domain@localhost:9990 /] /host=master/core-service=management/access=audit:read-resource(recursive=true)
{
    "outcome" => "success",
    "result" => {
        "file-handler" => {
            "host-file" => {
                "formatter" => "json-formatter",
                "max-failure-count" => 10,
                "path" => "audit-log.log",
                "relative-to" => "jboss.domain.data.dir"
            },
            "server-file" => {
                "formatter" => "json-formatter",
                "max-failure-count" => 10,
                "path" => "audit-log.log",
                "relative-to" => "jboss.server.data.dir"
            }
        },
        "json-formatter" => {"json-formatter" => {
            "compact" => false,
            "date-format" => "yyyy-MM-dd HH:mm:ss",
            "date-separator" => " - ",
            "escape-control-characters" => false,
            "escape-new-line" => false,
            "include-date" => true
        }},
        "logger" => {"audit-log" => {
            "enabled" => false,
            "log-boot" => true,
            "log-read-only" => false,
            "handler" => {"host-file" => {}}
        }},
        "server-logger" => {"audit-log" => {
            "enabled" => false,
            "log-boot" => true,
            "log-read-only" => false,
            "handler" => {"server-file" => {}}
        }},
        "syslog-handler" => undefined
    }
}

We now have two file handlers, one called host-file used to configure the file to log management operations on the host, and one called server-file used to log management operations executed on the servers. Then logger=audit-log is used to configure the logger for the host controller, referencing the host-file handler. server-logger=audit-log is used to configure the logger for the managed servers, referencing the server-file handler. The attributes for server-logger=audit-log are the same as for server-logger=audit-log in the previous section. Having the host controller and server loggers configured independently means we can control audit logging for managed servers and the host controller independently. 

Canceling management operations

WildFly 9 includes the ability to use the CLI to cancel management requests that are not proceeding normally.

The cancel-non-progressing-operation operation

The cancel-non-progressing-operation operation instructs the target process to find any operation that isn't proceeding normally and cancel it.

On a standalone server:

[standalone@localhost:9990 /] /core-service=management/service=management-operations:cancel-non-progressing-operation
{
    "outcome" => "success",
    "result" => "-1155777943"
}

The result value is an internal identification number for the operation that was cancelled.

On a managed domain host controller, the equivalent resource is in the host=<hostname> portion of the management resource tree:

[domain@localhost:9990 /] /host=host-a/core-service=management/service=management-operations:cancel-non-progressing-operation
{
    "outcome" => "success",
    "result" => "2156877946"
}

An operation can be cancelled on an individual managed domain server as well:

[domain@localhost:9990 /] /host=host-a/server=server-one/core-service=management/service=management-operations:cancel-non-progressing-operation
{
    "outcome" => "success",
    "result" => "6497786512"
}

An operation is considered to be not proceeding normally if it has been executing with the exclusive operation lock held for longer than 15 seconds. Read-only operations do not acquire the exclusive operation lock, so this operation will not cancel read-only operations. Operations blocking waiting for another operation to release the exclusive lock will also not be cancelled.

If there isn't any operation that is failing to proceed normally, there will be a failure response:

[standalone@localhost:9990 /] /core-service=management/service=management-operations:cancel-non-progressing-operation
{
    "outcome" => "failed",
    "failure-description" => "WFLYDM0089: No operation was found that has been holding the operation execution write lock for long than [15] seconds",
    "rolled-back" => true
}

The find-non-progressing-operation operation

To simply learn the id of an operation that isn't proceeding normally, but not cancel it, use the find-non-progressing-operation operation:

[standalone@localhost:9990 /] /core-service=management/service=management-operations:find-non-progressing-operation
{
    "outcome" => "success",
    "result" => "-1155777943"
}

If there is no non-progressing operation, the outcome will still be success but the result will be undefined.

Once the id of the operation is known, the management resource for the operation can be examined to learn more about its status.

Examining the status of an active operation

There is a management resource for any currently executing operation that can be queried:

[standalone@localhost:9990 /] /core-service=management/service=management-operations/active-operation=-1155777943:read-resource(include-runtime=true)
{
    "outcome" => "success",
    "result" => {
        "access-mechanism" => "undefined",
        "address" => [
            ("deployment" => "example")
        ],
        "caller-thread" => "management-handler-thread - 24",
        "cancelled" => false,
        "exclusive-running-time" => 101918273645L,
        "execution-status" => "awaiting-stability",
        "operation" => "deploy",
        "running-time" => 101918279999L
    }
}

The response includes the following attributes:

Field

Meaning

access-mechanism

The mechanism used to submit a request to the server. NATIVE, JMX, HTTP

address

The address of the resource targeted by the operation. The value in the final element of the address will be '<hidden>' if the caller is not authorized to address the operation's target resource.

caller-thread

The name of the thread that is executing the operation.

cancelled

Whether the operation has been cancelled.

exclusive-running-time

Amount of time in nanoseconds the operation has been executing with the exclusive operation execution lock held, or -1 if the operation does not hold the exclusive execution lock.

execution-status

The current activity of the operation. See below for details.

operation

The name of the operation, or '<hidden>' if the caller is not authorized to address the operation's target resource.

running-time

Amount of time the operation has been executing, in nanoseconds.

The following are the values for the exclusive-running-time attribute:

Value

Meaning

executing

The caller thread is actively executing

awaiting-other-operation

The caller thread is blocking waiting for another operation to release the exclusive execution lock

awaiting-stability

The caller thread has made changes to the service container and is waiting for the service container to stabilize

completing

The operation is committed and is completing execution

rolling-back

The operation is rolling back

All currently executing operations can be viewed in one request using the read-children-resources operation:

[standalone@localhost:9990 /] /core-service=management/service=management-operations:read-children-resources(child-type=active-operation)
{
    "outcome" => "success",
    "result" => {"-1155777943" => {
        "access-mechanism" => "undefined",
        "address" => [
            ("deployment" => "example")
        ],
        "caller-thread" => "management-handler-thread - 24",
        "cancelled" => false,
        "exclusive-running-time" => 101918273645L,
        "execution-status" => "awaiting-stability",
        "operation" => "deploy",
        "running-time" => 101918279999L
    },
    {"-1246693202" => {
        "access-mechanism" => "undefined",
        "address" => [
            ("core-service" => "management"),
            ("service" => "management-operations")
        ],
        "caller-thread" => "management-handler-thread - 30",
        "cancelled" => false,
        "exclusive-running-time" => -1L,
        "execution-status" => "executing",
        "operation" => "read-children-resources",
        "running-time" => 3356000L
    }}
}

Canceling a specific operation

The cancel-non-progressing-operation operation is a convenience operation for identifying and canceling an operation. However, an administrator can examine the active-operation resources to identify any operation, and then directly cancel it by invoking the cancel operation on the resource for the desired operation.

[standalone@localhost:9990 /] /core-service=management/service=management-operations/active-operation=-1155777943:cancel
{
    "outcome" => "success",
    "result" => undefined
}

Controlling operation blocking time

As an operation executes, the execution thread may block at various points, particularly while waiting for the service container to stabilize following any changes. Since an operation may be holding the exclusive execution lock while blocking, in WildFly 9 execution behavior was changed to ensure that blocking will eventually time out, resulting in roll back of the operation.

The default blocking timeout is 300 seconds. This is intentionally long, as the idea is to only trigger a timeout when something has definitely gone wrong with the operation, without any false positives.

An administrator can control the blocking timeout for an individual operation by using the blocking-timeout operation header. For example, if a particular deployment is known to take an extremely long time to deploy, the default 300 second timeout could be increased:

[standalone@localhost:9990 /] deploy /tmp/mega.war --headers={blocking-timeout=450}

Note the blocking timeout is not a guaranteed maximum execution time for an operation. If it only a timeout that will be enforced at various points during operation execution.

Configuration file history

The management operations may modify the model. When this occurs the xml backing the model is written out again reflecting the latest changes. In addition a full history of the file is maintained. The history of the file goes in a separate directory under the configuration directory.

As mentioned in Command line parameters#parameters the default configuration file can be selected using a command-line parameter. For a standalone server instance the history of the active standalone.xml is kept in jboss.server.config.dir/standalone_xml_history (See Command line parameters#standalone_system_properties for more details). For a domain the active domain.xml and host.xml histories are kept in jboss.domain.config.dir/domain_xml_history and jboss.domain.config.dir/host_xml_history.

The rest of this section will only discuss the history for standalone.xml. The concepts are exactly the same for domain.xml and host.xml.

Within standalone_xml_history itself following a successful first time boot we end up with three new files:
*standalone.initial.xml - This contains the original configuration that was used the first time we successfully booted. This file will never be overwritten. You may of course delete the history directory and any files in it at any stage.
*standalone.boot.xml - This contains the original configuration that was used for the last successful boot of the server. This gets overwritten every time we boot the server successfully.
*standalone.last.xml - At this stage the contents will be identical to standalone.boot.xml.  This file gets overwritten each time the server successfully writes the configuration, if there was an unexpected failure writing the configuration this file is the last known successful write.

standalone_xml_history contains a directory called current which should be empty. Now if we execute a management operation that modifies the model, for example adding a new system property using the CLI:

[standalone@localhost:9999 /] /system-property=test:add(value="test123")
{"outcome" => "success"}

What happens is:

  • The original configuration file is backed up to standalone_xml_history/current/standalone.v1.xml. The next change to the model would result in a file called standalone.v2.xml etc. The 100 most recent of these files are kept.

  • The change is applied to the original configuration file

  • The changed original configuration file is copied to standalone.last.xml

When restarting the server, any existing standalone_xml_history/current directory is moved to a new timestamped folder within the standalone_xml_history, and a new current folder is created. These timestamped folders are kept for 30 days.

Snapshots

In addition to the backups taken by the server as described above you can manually take take snapshots which will be stored in the snapshot folder under the _xml_history folder, the automatic backups described above are subject to automatic house keeping so will eventually be automatically removed, the snapshots on the other hand can be entirely managed by the administrator.

You may also take your own snapshots using the CLI:

[standalone@localhost:9999 /] :take-snapshot
{
    "outcome" => "success",
    "result" => {"name" => "/Users/kabir/jboss-7.0.0.CR1/standalone/configuration/standalone_xml_history/snapshot/20110630-172258657standalone.xml"}
}

You can also use the CLI to list all the snapshots

[standalone@localhost:9999 /] :list-snapshots
{
    "outcome" => "success",
    "result" => {
        "directory" => "/Users/kabir/jboss-7.0.0.CR1/standalone/configuration/standalone_xml_history/snapshot",
        "names" => [
            "20110630-165714239standalone.xml",
            "20110630-165821795standalone.xml",
            "20110630-170113581standalone.xml",
            "20110630-171411463standalone.xml",
            "20110630-171908397standalone.xml",
            "20110630-172258657standalone.xml"
        ]
    }
}

 To delete a particular snapshot:

[standalone@localhost:9999 /] :delete-snapshot(name="20110630-165714239standalone.xml")
{"outcome" => "success"}

and to delete all snapshots:

[standalone@localhost:9999 /] :delete-snapshot(name="all")
{"outcome" => "success"}

In domain mode executing the snapshot operations against the root node will work against the domain model. To do this for a host model you need to navigate to the host in question:

[domain@localhost:9999 /] /host=master:list-snapshots
{
    "outcome" => "success",
    "result" => {
        "domain-results" => {"step-1" => {
            "directory" => "/Users/kabir/jboss-7.0.0.CR1/domain/configuration/host_xml_history/snapshot",
            "names" => [
                "20110630-141129571host.xml",
                "20110630-172522225host.xml"
            ]
        }},
        "server-operations" => undefined
    }
}

Subsequent Starts

For subsequent server starts it may be desirable to take the state of the server back to one of the previously known states, for a number of items an abbreviated reverence to the file can be used: -

Abreviation

Parameter

Description

initial

--server-config=initial

This will start the server using the initial configuration first used to start the server.

boot

--server-config=boot

This will use the configuration from the last successful boot of the server.

last

--server-config=last

This will start the server using the configuration backed up from the last successful save.

v?

--server-config=v?

This will server the _xml_history/current folder for the configuration where ? is the number of the backup to use.

???? - ?????

--server-config=???? - ?????

The server will be started after searching the snapshot folder for the configuration which matches this prefix.

In addition to this the --server-config parameter can always be used to specify a configuration relative to the jboss.server.config.dir and finally if no matching configuration is found an attempt to locate the configuration as an absolute path will be made.

JBoss.org Content Archive (Read Only), exported from JBoss Community Documentation Editor at 2020-03-13 13:54:17 UTC, last content change 2014-07-03 04:30:31 UTC.